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(54) Abstract Title 

Echo Canceller with NLP processor 

(57) A nonlinear processor (NLP) for selectively removing or reducing residual echo signals from an acoustic 
echo canceller associated with a telephony terminal is provided Low level background noise and near end 
speech signals pass through the NLP structure substantially unaltered. Distortion, background noise above a 
preset threshold and echo signals including long duration echoes are replaced with a linear combination of 
previous noise data. 
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Nonlinear Processor for Acoustic Echo Canceller 



Field of Invention 

This invention pertains to the field of adaptive speech echo 
cancellation, and more particularly to acoustic echo 
cancellation for speaker-phones and voice conferencing 
systems utilizing a nonlinear processor. 



Background 

Nonlinear processors (NLPs) are used in echo cancellation 
generally, and in particular for echo cancellation of 
acoustic speech signals. Speech echo cancellation can be 
grouped into 2 major categories, network echo cancellation 
and acoustic echo cancellation. The primary difference 
between acoustic echo signals and network echo signals is 
that an acoustic echo channel includes loudspeaker and 
microphone transducers that convert signals to and from 
audible (acoustic) sound signals, as opposed to network echo 
signals that are generated by electric circuits (hybrids) . 
The acoustic type typically has high background noise 
signals present from the surrounding environment that makes 
application of prior art nonlinear processors unfavorable. 

Prior Art 

The term "nonlinear processor" or NLP can be used to 
describe a signal processing circuit or algorithm that is 
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placed in the speech path after echo cancellation, so as to 
provide further attenuation or removal of residual echo 
signals that cannot be cancelled completely by an echo 
canceller. A non-linearity, a distortion, or an added noise 
5 signal are examples of signals that can r.ot be cancelled by 
an echo car.celler, and these signals are typically removed 
or attenuated by a nonlinear processor. 

One example of a prior art NLP is a " centre clipper" in 
10 which all signal samples with amplitude less than a 

threshold value are set to zero. This method has been used 
for network echo cancellation for many years by many 
different equipment suppliers. A description of the 
operation of such an NLP has been included in the appendix 

15 of the ITU-T G.165 recommendation as a reference design for 
an NLP. A <nown problem with this type of NLP is the so 
called "ncise gating" phenomena wherein a party listening to 
the resulting speech signals, after a centre clipping NLP, 
hears the background noise signals disappearing and then re- 

20 appearing during periods of activation and de-act ivat ion of 
the NLP. 

improvements upon this centre clipper method that reduce or 
eliminate the "noise gating" problem have been introduced in 
25 recent years. These improvements are primarily used lr>r 
network type echo cancellers in which background noiso 
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levels are typically very low in comparison to the noise 
levels experienced with acoustic echo signals. An example of 
a prior art NLP improvement is a centre clipper method 
combined with the injection of a matched artificial noise 
source to mask the removal of noise signals by the centre 
clipper. Yet another example is a variable attenuator that 
provides a soft-switched transition between on/off states of 
signal attenuation with complementary soft-switched 
injection of artificial noise. US Patent 5,274,705, which 
issued December 28, 1993 to Younce et al, describes another 
example of an improved NLP using dual thresholds in the NLP 
transfer function which allows transparent transfer of low 
level noise signals if below the low threshold, and 
transparent transfer of large signals if above an upper 
threshold while removing or modifying any signals in-between 
the two thresholds. 

Problems with all of the aforementioned methods arise wnen 
dealing with signals from an acoustic environment because of 
the higher noise levels. Noise injection methods are net 
typically used because the character of the background r.oise 
changes very noticeably if an artificial noise is injected 
in place of the original noise. Variable attenuation methods 
without noise injection appear "to be most commonly use- for 
the control of residual echo in acoustic echo cancellers. 
This appears to be an extension of methods used prev:--.sly 
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by half-duplex speakerphones and network echo suppressors 
which used complementary attenuators to provide switched 
loss to control echo. The use of echo cancellation for a 
"full duplex" handsfree telephone appears to also make use 

5 of prior art complementary attenuators with reduced attenu- 
ation "depth" to make the connection close to full duplex, 
or perhaps, subjectively, "full duplex". Some other 
implementations appear to allow complete full-duplex 
communicaticr. some of the time (e.g. during double-talk 

10 periods), while providing some extra attenuation control of 
echo residual during other periods of tine (e.g. single talk 
periods). All of these methods cause audible changes in 
background r.oise signals producing some degradation of 
overall subjective performance. 



15 



The prior art dual threshold method when applied to acoustic 
background r.oise signals, produces noticeable levels of 
extra signal distortion. This distortion is caused by the 
changes made to signals when the NLP is on. This audible 

20 distortion changes the character of the background noise 
during speech from the far end side, and can best be 
described as a raspy type noise with some high frequency 
components that sound different than a typical background 
noise. Note as used in this description the far end talker 

25 is the party who is also listening to the resulting signal 
after the NIP. 
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Another problem with the prior art NLP is that it has no 
control over a long echo path environment. To save cost most 
echo cancellers can only deal with a short echo length (e.g. 
128 ms or less) . In some acoustic environments, the echo can 

5 last for about 0.5 to 1 sec. Although, in most cases, the 

echo residual is very small after 128 ms, when both sides of 
telephone line are quiet, even a very small echo residual is 
noticeable. After the loudspeaker has been quiet for over 
1/2 sec, the echo may still be present at the microphone 

10 input. The echo residual is treated as near-end single talk 
by the speaker-phone, and therefore the NLP will not 
attenuate this signal. 

Summary of the Invention 
15 The method used in the present invention builds upon the 
dual threshold method. The NLP turns on only if both a 
double talk condition and an echo suppression requirement 
are met. 

20 The present invention further relates to a method of 

reducing the level of extra signal distortion by processing 
signals in a different manner than the methods described in 
prior art NLP designs. The signal will be transparent if it 
is detected to be noise, otherwise a noise prediction value 

25 is sent out . 
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in the present invention, the long echo residual is dealt 
with by the new NLP structure. In lab tests, the echo 
residual is significantly reduced with the r.ew NLP 
structure, ever, in the case when echo signals last up to 1 
5 sec. and the adaptation algorithm can only deal with 100 ms 
echo length. 

Briefly, the NLP structure of the present invention 
determines whether the residual signal from the echo 

10 canceller is greater or less than an estimated noise level. 
If it is less than the estimated noise level the residual 
signal is passed through the NLP substantially unchanged. If 
the residual signal is greater than the estimated noise 
level it is further evaluated to determine whether or not it 

15 represents a near-end speech signal. If it is speech as in 
near-end single talk or double talk the residual signal is 
again passed through the NLP unchanged. If, however the 
incoming signal is echo residual or long term echo the NLP 
outputs a low level noise signal which represents a 

20 prediction based on previous noise samples. 

Therefore in accordance with a first aspect of the present 
invention there is provided a non linear processor (NLP) for 
use with an acoustic echo canceller associated with a 
25 telephone terminal to selectively reduce residual sigr.als 
therefrom. The NLP comprises: a first input to receive the 
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residual signal; a second input to receive a reference 
signal representing a signal from a far end user; a third 
input for receiving a near end signal from a microphone in 
the terminal; an output for delivering a NLP output to a far 

5 end user; a NLP switch, switchable between a first position 
wherein the residual signal is passed directly to the output 
and a second position wherein a signal representing a 
previous noise signal is delivered to the output; noise 
decision means to determine whether the residual signal is 

10 above a noise level and if not to switch the NLP switch to 
the first position; and NLP decision means cooperating with 
the noise decision means to switch the NLP switch to the 
first position when the residual signal contains near end 
speech and to the second position otherwise. 

15 

In a preferred embodiment the decision means incorporates an 
echo suppression threshold means which determined whether 
the residual signal is a long echo which was not cancelled 
by the echo canceller. If it is a long echo the switch 
20 remains in the second position wherein low level noise data 
is provided to Sout. 

In accordance with a second aspect of the present invention 
there is provided a method of selectively reducing a. 
25 residual signal from an acoustic echo canceller associated 
with a telephone terminal. The method comprises: 



10 



20 
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providing the residual signal to noise decision means 
for comparison with an estimated noise level; 

passing the residual signal directly through the NLP if 
it is less thar. the estimated noise level; 

passing tr.e residual signal to further decision means 
if it is greater than the estimated noise level whereat the 
residual signal is caused to be passed through the NLP if it 
is a near end speech signal otherwise a signal representing 
a previous noise signal is output from the NLP. 



Brief Description of the Drawings 

The invention will now be described 'in greater detail having 

reference to the attached drawings wherein: 

Figure 1 shows a typical acoustical echo car.celler with an 

15 incorporated NIP; 

Figure 2 is a block diagram of the NLP structure according 

to the present invention; and 

Figures 3A and 3B illustrate a flow diagram of the NLP 
process of the invention. 



25 



an 



Detailed Description of the Invention 

Fig. 1 shows a typical acoustical echo canceller having 
incorporated NLP . In Fig. 1, S(n) is the near-end sigr.al, 
R(n) is the far-end signal used as reference signal for the 
adaptive filter, and E(n) is the echo residual which is the 
difference between S(n) and its estimation value Sir.-. 
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Fig. 2 shows the implementation of the new NLP structure, 
which is composed of four parts: the main NLP block for 
signal input and output; the NOISE DECISION circuit to check 
whether the input signal is noise or an active signal; the 
5 NLP CONTROL to turn the NLP "ON" and "OFF" ; and the ECHO 

TAIL CONTROL to check whether the switch of NLP from "ON" to 
"OFF" is caused by the near end signal or the echo residue 
being too long to be cancelled by the adaptive filter. 

10 As shown in Figure 2 the echo residual signal E(n) is 

supplied to the NLP block and to the noise decision circuit. 
The NLP block includes switches 6 and 4 and filter 5. The 
activation of switch 6 is controlled by the output of AND 
Gate 7 such that when the output of Gate 7 is "1" the output 

15 of switch 6 is provided by the filter 5 i.e.S-1 and when the 
output of AND Gate 7 is "0" the output of switch 6 is a 
direct passthrough of E(n). This is shown in Figures 2 as 
S=0. Switch 4 is controlled by the output of comparator 3 
such that a "1" to switch 4 causes the switch to provide a 

20 "0" input to filter 5 and a "0" to switch 4 causes E ( n } to 
be supplied to filter 5. 

The noise decision block includes ABS 1, noise level 
detector 2 and comparator 3. The purpose of the noise 
25 decision block is to monitor the residual echo with 

reference to an estimated noise level. When the lev-?: sf 
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E(n) is lower than the noise level (T no i S e) the output of 
comparator 3 is a"0". The noise level can be estimated with 
any common, noise-level detection algorithm in olock 2 whose 
output is T nol9 e. The "0"at the output of comparator 3 is 
5 supplied to one input of AND gate 7 which switches switch 6 
to S=0 and as previously discussed the residual echo signal 
E(n)is passed directly through the NLP structure. Thus, any 
residual echo whose level is lower than a predetermined 
noise threshold is not altered by the NLP. This "0" at the 
10 output of comparator 3 is also provided to switch 4 so that 
in accordance with the previous discussion E(n) is also 
supplied to the input of filter 5. Because switch 6 is in 
position 3=0, E(n) is not connected to Sout but rather is 
the output of filter 5. The filter is normally a low-cost 
15 FIR filter with low-pass characteristics. It takes the 

noise samples in E(n) signal, smooths them and subsequently 
outputs them as a noise predicted value. 

If the value of E(n) is greater than the predetermined no.se 
threshold the output of comparator 3 is a "1" and this "1" 
20 appears at one input of AND gate 7 as well as -_o switch 4 

thereby switching S4 to S-l . In this configuration switch 4 
receives the "0" input which is supplied to filter 5. The 
operation of switch 6, in this mode, is now dependent or. the 
NLP decision coming out of OR "gate 22. As illustrated in 
25 Figure 2, OR gate 22 has two inputs, one from NLP control 
and one from the echo tail control. 
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Looking first at the NLP control block which has two 
comparator circuits, one for double talk detection and the 
other for a situation wherein the echo canceller shown in 
Figure 1 does not provide enough echo cancellation. This 
could be because of long echo, because the adaptive 
algorithm does not converge sufficiently or because of a 
small echo with a small double talk. The double talk 
comparator circuit includes level detector 12, loss 
threshold 10, comparator 9 and hangover timer 11. When the 
value of E(n) is greater than the noise threshold but is not 
near-end speech the value of E(n) will be less than the 
level of R(n) which is multiplied by a loss threshold. 
Under these conditions the output of comparator 9 will be a 
"1" which is supplied to one input of AND gate 15. Under 
the same conditions, i.e. no near-end speech, the value of 
E(n) is smaller than the value of S(n) multiplied by a 
suppression loss T sup and in this situation the output of 
comparator 8 is a "1". Thus the output of AND gate 15 is 
also a "1" and hence the NLP decision is a "1" which, in 
turn means that switch 6 is in the position S=l and the 
output of the NLP structure is a filtered value of a 
previous noise sample. Thus any residual echo is reduced or 
removed from the signal by the NLP before it is sent to a 
fa r-end user . 
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If double talk occurs i.e. the far-end speaker is talking 
and the near-end speaker talks as well, the signal E(n) now 
represents active voice communication and is to be passed 
directly through the NLP structure. When there ,s a double 
talk situation the signal at the negative input of 
comparator 9 rises above the level of R(n) multiplied by 
Tloss and the output of comparator 9 switches to a "0". 
Hangover timer 11 simply delays for a preselected interval 
the switchover from a "0" to a "1" to extend the detect time 
of double talk.. in any event, a "0" on one of the inputs 
to AND gate 15 results in a "0" being provided to one of the 
inputs to OR gate 22. Under normal circumstances the output 
of AND gate 21 in the echo tail control will also be a "0" 
so that AND gate 1 will also switch to a "0" output 
resulting in S6 switching to S=0 and Sout-E(n). Thus, the 
residual echo which now includes speech from a r.ear-end user 
is passed through the NLP structure unaltered. 

Another scenario which might arise is when the far-end user 
is silent but the near-end user is speaking i.e. near-end 
single talk, again this residual signal is to be passed 
through the NLP structure without alteration. This 
situation is covered by the aforementioned structure ar.d the 
structure comprising level detector 14, echo suppression 
threshold 13, and comparator 8. In this situation tr.e level 
of S(n) multiplied with T sup drops below the level of r. r.) 
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and the output of comparator 8 switches from a "1" to a "0". 
This rt 0" on AND gate 15 results in a "0" to one input of OR 
gate 22 and again, providing the output of echo tail AND 
gate 22 is a "0", switch 6 is switched to S=0 and the value 
of E(n) is provided to Sout. 

There is one additional condition which must be considered 
and that is the situation wherein the near-end signal 
appears to be near-end speech but is, in fact, a long 
duration echo such as might occur with a speaker phone or 
the like. The adaptive filter in the echo canceller shown 
in Figure 1 and as discussed previously normally only 
operates on a short echo length e.g. 128 ms . or less. An 
echo which lasts longer than this time interval will appear 
in residual echo signal E(n) and without the benefit of the 
echo tail control of the present invention would be passed 
through the structure on the false decision that it 
represents near-end speech. Thus, when comparator 8 
switches from "1" to a "0" output indicating near-end 
speech, the output from AND gate 15 to OR gate 22 is a "0". 
At this time, the echo tail control block comprising 
residual level delay 19, threshold 17, NLP decision delay 20 
and comparator 18 determine whether the current value of 
E(n), i.e. P* in Figure 2 is greater or less than a previous 
value of P*i.e. P k -i. If the previous value, P k -i, (with a 
threshold y) is greater than P k which would suggest a 
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decaying signal, i.e. a long-term echo, comparator 18 
outputs a "1". Since the output of OR gate 22 is also a "1" 
from the previous time, this "1" is supplied through 
decision delay block 20 to one input of AND gate 21. The 
other input of AND gate 21 is also a "1" by virtue of the 
output of comparator 18. Thus, OR gate 22 continues to 
output a "1" so that Sout is the filtered noise value rather 
than E(n) when E(n) is above the noise level. When the 
value of E(n) rises such that P k is greater or equal to 
multiplied by y. comparator 18 switches to a "0" output and 
as a result NLP decision will become "0" and E(n) will again 
pass directly through to Sout. This rise in E(N) could, for 
example, indicate a situation wherein there is a near-end 
speaker and/or a double talk situation. 

The echo tail control block provides the added functionality 
of removing echoes having a long tail which would otherwise 
be passed through the NLP structure on the basis that it was 
misinterpreted as being a near-end speech. 

Figure 3A and 3B is a flowchart setting out the process 
steps followed by the NLP structure. 

According to the present invention various alternatives nay 
be introduced. For example, P, may choose not to be updated 
when NLP control is "0"and NLP decision is "1" which mear.s 
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that the NLP is "ON" because of a long echo tail. The 
advantage of that is that P.-a will not be decreased during 
the echo tail and it gives a better chance for NLP to remain 
-ON" to combat a very long echo tail. The NLP will not be 
released with an occasional level reduction during the echo 
tail period. The disadvantage is that it may take a little 
longer to release NLP when both sides of the telephone line 
are quiet. 

Also, all the level calculations can be replaced with energy 
calculations. The disadvantage of that is that the energy 
responds slowly in comparison with peak level. 

The following sets out some of the parameter selections for 

the NLP configuration. 

1. Threshold for NLP tail decision ( Y ) : Large y will roake it 
difficult to release NLP when both sides of telephone are 
quite. On the other hand, small y will make it difficult to 
detect echo tail because the level of echo tail may not 
decrease strictly monotonically . In some cases, the residua 
level can be occasionally increased during the echo tail 
period and NLP can be turned off by these level increase if 
y is too small. A suitable value for y in the acoustical 
echo cancellation is 1.05. 
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2. The function of the filter is to replace the missing 
noise samples. In the acoustic echo environment, the 
background noise is not white but coloured with low pass 
characteristics. Therefore, a low pass filter should be used 
to recover noise samples. A simple and efficient filter is 4 
taps FIR filter with its first coefficient being zero: [0, 
0.29469694, 0.34868972, 0.20388524]. 

3. The double talk threshold (T lo «> should also be chosen 
carefully. If it is too large, double talk may not be 
detected efficiently and if it is too small, NLP may not 
function well because the double talk detector may give a 
lot of false double talk indication. A suitable value for 
Tioss is 0.5. 

4. The chosen criteria for the double talk hangover timer is 
the same as double talk threshold. If it is too small, the 
double talk detector may not work well and a lot of near-end 
speech clipping can be heard by the far-end listeners. If 
the hangover timer is too large, it takes a long time to 
release the double talk decision and NLP may not function 
well to cut the echo residual effectively. A suitable value 
for the hangover timer is 400 samples. 

The threshold for echo suppression (T sup ) may have a 
relatively large range. It is a safe protection for t r - 
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small near end double talk. A very small near-end double 
talk may not be detected by the double talk detector, but it 
will seriously deteriorate the echo canceller performance, 
in such a case, an echo suppression level detection should 

5 be employed. A high echo suppression threshold will imply 
that small double talk in the echo environment may not be 
detected effectively and a low threshold means that NLP will 
not turn on easily. With a very low threshold, it will be 
difficult or at least take a long time to turn the NLP on 

10 because the NLP will be activated only when large amount of 
echo suppression is achieved by the adaptive echo canceller. 
A suitable value for the threshold is T sup is 0.2. 

The following provides some definitions which may assist in 
15 an understanding of the invention. 

NLP: Nonlinear processor, used to remove or further 
attenuate residual echo signals after echo cancellation. 

20 Adaptive Filter: An adaptive algorithm to simulate the echo 
path so that the echo can be removed by subtracting its 
estimated value. 

Double-Talk Detector: detects'the condition of double-talk 
25 (when both the near-end and the far-end signals exist). 
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Level Detector: A recursive algorithm to detect the peak 
averaged value of the signal. 

Noise Level Detector: A recursive algorithm to estimate the 
5 level of background noise. 

While a particular embodiment of the invention has been 
described and illustrated it will be apparent to one skilled 
in the art that numerous variations can be made to the basic 
10 concept.. It is to be understood, however, that such 

variations will fall within the scope of the invention as 
defined by the appended claims. . 
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Clains : 

l.A non linear processor (NLP) for use with an acoustic 
echo canceller associated with a telephone terminal to 
selectively reduce residual signals from said echo 
5 canceller, said NLP comprising: 

a first input to receive said residual signal; 
a second input to receive a reference signal 
representing a signal from a far end user; 

a third input for receiving a near end signal from a 
10 microphone in said terminal 

an output for delivering a NLP output to a far end 

user; 

a NLP switch, switchable between a first position 
wherein said residual signal is passed directly to said 
,5 output and a second position wherein a signal representing a 
previous noise signal is delivered to said outputs- 
noise decision means to determine whether said residual 
signal is above a noise level and if not to switch said NLP 
switch to said first position; and 
20 NLP decision means cooperating with said noise decision 

means to switch said NLP switch to said first position when 
said residual signal contains near end speech and to said 
second position otherwise. 
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2. A NLP as defined in claim 1 having a filter means to 
dynamically recalculated said previous noise signal when 
said residual signal is greater than said noise level. 

3. A NLP as defined in claims 1 or 2 wherein said NLP 
decision means includes means to compare said residual 
signal with said reference signal and if said residual 
signal is greater than a set reference signal level thereby 
indicating a double talk situation to cause said NLP switch 
to switch to said first position. 

4. A NLP as defined in claim 3 further including timer 
means to delay switching of said NLP switch until after a 
preset time interval has passed. 

5. A NLP as defined in claims 1 or 2 wherein said NLP 
decision means includes means to compare said residual 
signal and said near end signal and if said residual signal 
is larger than a preset near end signal level to switch said 
NLP switch to said first position. 

6. A NLP as defined in claim 5 further having means to 
determine whether said residual signal is a long echo ar.d if 
so to maintain said switch in said second position.- 
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7. A method of selectively reducing a residual signal 
from an acoustic echo canceller associated with a telephone 
terminal, s~id method comprising: 

providing said residual signal to ncise decision means 
5 for comparison with an estimated noise level; 

passing said residual signal directly through said NLP 
if it is less than said estimated noise level; 

passing said residual signal to further decision means 
if it is greater than said estimated noise level f whereat 
10 said residual signal is caused to be passed through said NLP 
if it is a near end speech signal otherwise a signal 
representing a previous noise signal is output from said 
NLP . 

15 8. A system substantially as herein described, with 

reference to Figure 2 of the accompanying drawings . 

9. A method substantially as herein described, with 
reference to Figures 3A and 3B of the accompanying drawings. 
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